Step 3: Integrating LLM: Using Llama 3 in WatsonX as LLM

Running simple LLM

Let's start by generating text with LLMs. Create a Python file and name it simple_llm.py. You can proceed by clicking the link below or by referencing the accompanying image.

In case, you want to use Llama 3 as an LLM instance, you can follow the instructions below:

IBM WatsonX utilizes various language models, including Llama 3 by Meta, which is currently the strongest open-source language model.

Here's how the code works:

  1. Setting up credentials: The credentials needed to access IBM's services are pre-arranged by the Skills Network team, so you don't have to worry about setting them up yourself.

  2. Specifying parameters: The code then defines specific parameters for the language model. 'MAX_NEW_TOKENS' sets the limit on the number of words the model can generate in one go. 'TEMPERATURE' adjusts how creative or predictable the generated text is.

  3. Setting up Llama 3 model: Next, the LLAMA3 model is set up using a model ID, the provided credentials, chosen parameters, and a project ID.

  4. Creating an object for Llama 3: The code creates an object named llm, which is used to interact with the Llama 3 model. A model object, LLAMA3_model, is created using the Model class, which is initialized with a specific model ID, credentials, parameters, and project ID. Then, an instance of WatsonxLLM is created with LLAMA3_model as an argument, initializing the language model hub llm object.

  5. Generating and printing response: Finally, 'llm' is used to generate a response to the question, "How to read a book effectively?" The response is then printed out.

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  1. from ibm_watson_machine_learning.foundation_models import Model
  2. from ibm_watson_machine_learning.foundation_models.extensions.langchain import WatsonxLLM
  3. from ibm_watson_machine_learning.metanames import GenTextParamsMetaNames as GenParams
  4. my_credentials = {
  5. "url" : "https://us-south.ml.cloud.ibm.com"
  6. }
  7. params = {
  8. GenParams.MAX_NEW_TOKENS: 700, # The maximum number of tokens that the model can generate in a single run.
  9. GenParams.TEMPERATURE: 0.1, # A parameter that controls the randomness of the token generation. A lower value makes the generation more deterministic, while a higher value introduces more randomness.
  10. }
  11. LLAMA2_model = Model(
  12. model_id= 'meta-llama/llama-3-2-11b-vision-instruct',
  13. credentials=my_credentials,
  14. params=params,
  15. project_id="skills-network",
  16. )
  17. llm = WatsonxLLM(LLAMA2_model)
  18. print(llm("How to read a book effectively?"))

You can then run this script in the terminal using the following command:

  1. 1
  1. python3 simple_llm.py

Upon running the script, you should see the generated text in your terminal, as shown below:

You can see how watsonx Llama 2 provides a good answer.